Large data sets may exist in various sizes and organizational structures. With big data comprising data sets as large as ever, the volume of data collected incident to the increased popularity of online and electronic transactions continues to grow. For example, billions of records (also referred to as rows) and hundreds of thousands of columns worth of data may populate a single table. The large volume of data may be collected in a raw, unstructured, and undescriptive format in some instances. However, traditional relational databases may not be capable of sufficiently handling the size of the tables that big data creates.
As a result, the massive amounts of data in big data sets may be stored in numerous different data storage formats in various locations to service diverse application parameters and use case parameters. Each different data storage format typically has a different interface approach as well. For users, the difficulty of learning the various interface protocols, each having varying query syntaxes and adapting programs to interact with multiple storage formats, creates difficulties for users of big data formats.
Application development is a complicated process and, in a big data environment, may tend to rely on dedicated personnel. Creating use cases to support production applications involves preparation of data, transformations of that data, actions upon the data, and interaction with other applications and platforms. The various tasks carried out to support application development may involve interfacing with various technologies for data storage and interfacing by various team members using various systems. The pool of development personnel qualified to use the various technologies may be limited.